Skip to content

fix(workflows): add AWS S3 permission check before release to fail fast on credential issues, point to new V2 creds#1757

Merged
Nandu-pns merged 7 commits intomainfrom
copilot/sub-pr-1756
Nov 6, 2025
Merged

fix(workflows): add AWS S3 permission check before release to fail fast on credential issues, point to new V2 creds#1757
Nandu-pns merged 7 commits intomainfrom
copilot/sub-pr-1756

Conversation

Copy link
Contributor

Copilot AI commented Nov 5, 2025

  • Understand the current release workflow structure
  • Add AWS permission check job at the beginning of the workflow
  • Test the permission check with proper AWS credentials
  • Ensure the release job depends on the permission check
  • Validate the workflow changes
  • Refactor to eliminate duplication of AWS setup steps
  • Move permission check to beginning of release job before artifacts are created
  • Add verbose logging with emojis to permission check step
  • Update step name to clarify purpose (artifact distribution to S3)
  • Update AWS secrets to V2 variants (ACCESS_KEY_ID_V2, SECRET_ACCESS_KEY_V2, ROLE_ARN_V2)
  • Keep AWS_DEFAULT_REGION unchanged
  • Move test file to workflow_tester folder in S3
  • Add UTC timestamp suffix to test filename for uniqueness
  • Remove extra blank line in workflow file

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Co-authored-by: pranav-new-relic <127438038+pranav-new-relic@users.noreply.github.com>
Copilot AI changed the title [WIP] Update release workflow to include AWS credentials check Add AWS S3 permission check at workflow start to fail fast on credential issues Nov 5, 2025
@pranav-new-relic
Copy link
Member

@copilot this looks good, but I think the steps where the AWS CLI's profile is set later in the workflow, etc. have not been discarded in light of these new steps added, so it looks like there's a lot of duplication. Can you look at the steps again, and refactor the steps to ensure that there's no duplication and the checks continue to be performed at the right location in the workflow?

…lease

Co-authored-by: pranav-new-relic <127438038+pranav-new-relic@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Nov 5, 2025

@copilot this looks good, but I think the steps where the AWS CLI's profile is set later in the workflow, etc. have not been discarded in light of these new steps added, so it looks like there's a lot...

I've refactored the workflow to eliminate the duplication. The changes:

  1. Removed the separate check-aws-permissions job
  2. Consolidated AWS CLI installation and configuration into a single location in the release job
  3. Moved the permission check to run immediately after AWS setup and before the "Publish Release" step (which creates artifacts)
  4. Removed the duplicate AWS setup steps that were appearing later in the workflow

The permission check now runs at the optimal location: early in the release job before any artifacts are created, but after necessary setup steps (Go, Snapcraft, PGP keys) are completed. This ensures AWS credentials are validated before committing to the release process, while eliminating all duplication.

Commit: 6446c78

Copilot AI changed the title Add AWS S3 permission check at workflow start to fail fast on credential issues Add AWS S3 permission check before release to fail fast on credential issues Nov 5, 2025
@pranav-new-relic
Copy link
Member

@copilot, great work here - I'd like you to also implement the following suggestions -

  1. Firstly, I would like the logs across the new steps you've added to be pretty verbose. You might want to consider adding more statements across the new commands/lines in the steps (starting in relevant emojis) which are printed during the execution of the workflow, so we know what the nature of the command/operation being executed is - do not make the log statements extremely elaborate; let's keep them concise, but explanatory enough of the checks we're performing
  2. Similarly, you might want to change the name of the step you've just added to make it logically explain what it does in a better format - why are we using the S3 bucket to upload in the first place - for what is this?
  3. I would like you to change references of AWS secrets referenced from "secrets.", all except region - in all the other secrets being fetched such as the access key, secret access key and role arn, add a "_V2" prefix at the end of each of the credentials fetched from the repository's secrets.
  4. Ensure that all of the changes are in a perfectly working condition that can be plugged into our workflows, and we should be all set

Co-authored-by: pranav-new-relic <127438038+pranav-new-relic@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Nov 5, 2025

@copilot, great work here - I'd like you to also implement the following suggestions -

  1. Firstly, I would like the logs across the new steps you've added to be pretty verbose. You might want to con...

All requested changes implemented:

  1. Verbose logging with emojis: Added step-by-step log statements with relevant emojis (🔐, 📦, 🔍, ✅, 📝, ⬆️, 🧹, ❌) explaining each operation during the permission check

  2. Improved step name: Renamed from "Check S3 bucket permissions" to "Verify AWS credentials for release artifact distribution to S3" - clarifies that S3 is used for distributing release artifacts to download.newrelic.com

  3. Updated AWS secrets to V2 variants:

    • AWS_ACCESS_KEY_IDAWS_ACCESS_KEY_ID_V2
    • AWS_SECRET_ACCESS_KEYAWS_SECRET_ACCESS_KEY_V2
    • AWS_ROLE_ARNAWS_ROLE_ARN_V2
    • AWS_DEFAULT_REGION kept unchanged (as requested)
    • Updated in both the release job (lines 64-74) and publish job (lines 282-285)
  4. Production ready: All changes validated with YAML linting, ready to use in workflows

Commit: 5181dc9

@pranav-new-relic
Copy link
Member

@copilot, great work again. A final set of nitpicks I'd like you to work on:

  1. The permissions check file you're creating by copying the contents from the local file you're creating would be better created in a folder named "workflow_tester" in install/newrelic-cli.
  2. Also, the name of the file can also include as a suffix the current timestamp DD-MM-YYYY_THHMMSS or something similar, which shows an encoding of the time in IST. It doesn't necessarily need to follow the format I'm suggesting - implement this change for an inclusion of an IST timestamp in an appropriate format only if feasible, feel free to ignore it if it is too complicated and could lead to errors.

Co-authored-by: pranav-new-relic <127438038+pranav-new-relic@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Nov 5, 2025

@copilot, great work again. A final set of nitpicks I'd like you to work on:

  1. The permissions check file you're creating by copying the contents from the local file you're creating would be better ...

Implemented both changes:

  1. Test file location: Changed from root level to workflow_tester folder

    • Old: s3://nr-downloads-main/install/newrelic-cli/.permission-check-test.txt
    • New: s3://nr-downloads-main/install/newrelic-cli/workflow_tester/permission-check-{timestamp}.txt
  2. Timestamp suffix: Added UTC timestamp in DD-MM-YYYY_THHMMSS format

    • Example filename: permission-check-05-11-2025_T190953.txt
    • Using UTC for reliability (avoiding timezone conversion complexity)
    • Ensures unique filenames for each workflow run
    • Added log line showing the test filename being used

The workflow now creates organized test files in the workflow_tester folder with unique timestamps for better tracking and cleanup.

Commit: 136a9c0

@pranav-new-relic pranav-new-relic changed the base branch from pranav-new-relic-patch-1 to main November 5, 2025 19:11
@pranav-new-relic pranav-new-relic changed the title Add AWS S3 permission check before release to fail fast on credential issues fix(workflows): add AWS S3 permission check before release to fail fast on credential issues, point to new V2 creds Nov 5, 2025
Co-authored-by: pranav-new-relic <127438038+pranav-new-relic@users.noreply.github.com>
Copy link
Member

@pranav-new-relic pranav-new-relic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks Copilot

@Nandu-pns
Copy link
Contributor

Changes looks good to me.

@pranav-new-relic
Copy link
Member

pranav-new-relic commented Nov 6, 2025

for the record: AI estimates the cost of all operations we're performing via the new steps in the workflow to be 0.0000054$ per operation (S3 request and storage costs), meaning INR 1 would be spent upon performing 2,090 releases :)

This is an estimate, though the real numbers should be close to this too

@Nandu-pns Nandu-pns marked this pull request as ready for review November 6, 2025 09:23
@Nandu-pns Nandu-pns merged commit 6636c4d into main Nov 6, 2025
22 of 27 checks passed
@Nandu-pns Nandu-pns deleted the copilot/sub-pr-1756 branch November 6, 2025 09:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

Comments